In the claims: 

Following is a complete set of claims as amended with this Response. 

1 . (Currently Amended) A method comprising: 

maintaining a current state of auxiliary information regarding a first frame of a 
sequence of video frames in storage in a local device hardware memory, the sequence of 
video frames being encoded as a video bit stream having video frame data for each 
respective video frame of the sequence of video frames , the auxiliary information 
comprising non-visual information regarding the video frames ; 
using an electronic comparator comparing the stored current state of auxiliary 
information with auxiliary information regarding a later current video frame of the 
sequence of video frames to determine differential information; and 
annotating the differential information to the video bit stream as an annotation to the 
video frame data for the current video frame only if the differential information indicates 
a change from the current state of the auxiliary information. 

2. (Original) The method of Claim 1, wherein the current state of auxiliary 
information indicates the latest encoded information in the bit stream and is maintained in 
a state storage memory. 

3. (Canceled). 

4. (Original) The method of Claim 1, wherein the auxiliary information 
comprises video processing information regarding video frames. 

5. (Original) The method of Claim 1, further comprising encoding the 
differential information before annotating the differential information. 

6. (Original) The method of Claim 5, wherein the encoded differential 
information comprises a list of parameters, the parameters being described in a tag 
lookup table. 

7. (Original) The method of Claim 1, wherein annotating the differential 
information comprises extending the video bit stream format to include the differential 
information. 

8. (Canceled) 
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9. (Original) The method of Claim 1, further comprising gathering the video 
information regarding the current video frame. 

10. (Original) The method of Claim 9, wherein the differential information 
comprises the difference between the current state of the auxiliary information and the 
gathered information. 

1 1 . (Currently Amended) A computer readable medium having stored thereon 
data representing instructions which, when executed by a computer, cause the computer 
to perform operations comprising: 

maintaining a current state of auxiliary information regarding a first frame of a sequence 
of video frames in storage in a local memory, the sequence of video frames being 
encoded as a video bit stream having video frame data for each respective video frame of 
the sequence of video frames , the auxiliary information comprising non-visual 
information regarding the video frames ; 

comparing the stored current state of auxiliary information with auxiliary information 
regarding a later current video frame of the sequence of video frames to determine 
differential information; 

annotating the differential information to the video bit stream as an annotation to the 
video frame data for the current video frame only if the differential information indicates 
a change from the current state of the auxiliary information. 

1 2. (Original) The medium of Claim 1 1 , wherein the current state of auxiliary 
information indicates the latest encoded information in the bit stream and is maintained in 
a state storage memory. 

13. (Canceled). 

14. (Original) The medium of Claim 1 1, further comprising encoding the 
differential information as a list of parameters, the parameters being described in a tag 
lookup table, before annotating the differential information. 

15. (Original) The medium of Claim 14, wherein annotating the differential 
information comprises extending the video bit stream format to include the differential 
information. 
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1 6 . ( Current 1 y Amended) An apparatus compris ing : 

a collector to receive auxiliary information regarding a first frame of a sequence of video 
frames and to maintain in storage a current state of the auxiliary information in a memory 
of the collector, the sequence of video frames being encoded as a video bit stream having 
video frame data for each respective video frame of the sequence of video frames , the 
auxiliary information comprising non-visual information regarding the video frames ; 
a comparator to receive the auxiliary information and compare the stored current state of 
the auxiliary information with auxiliary information regarding a later current video frame 
of the sequence of video frames to determine differential information; and 
an annotator to store the differential information in the video bit stream by extending the 
original bit stream format only if the differential information indicates a change from the 
current state of the auxiliary information. 

17. (Canceled). 

18. (Original) The apparatus of Claim 16, wherein the auxiliary information 
comprises video processing information regarding video frames. 

19. (Original) The apparatus of Claim 16, wherein the collector receives the 
sequence of video frames. 

20. (Original) The apparatus of Claim 16, further comprising a state storage 
memory coupled to the collector to store the current state of the auxiliary information. 

21. (Original) The apparatus of Claim 16, further comprising an encoder to 
encode the differential information as auxiliary information for the current frame data; 

22. (Original) The apparatus of Claim 16, wherein the encoded differential 
information comprises a list of parameters, the parameters being described in a tag 
lookup table. 

23. (Previously Presented) An apparatus comprising: 

a collector to receive auxiliary information regarding a first frame of a sequence of video 

frames and to maintain in storage a current state of the auxiliary information, the 

sequence of video frames being encoded as a video bit stream having video frame data 

for each respective video frame of the sequence of video frames; 

a state storage memory coupled to the collector to store the current state of the auxiliary 

information; 

a comparator to receive the auxiliary information and compare the stored current state of 
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the auxiliary information with auxiliary information regarding a later current video frame 
of the sequence of video frames to determine differential information; and 
a tag lookup table defining settable parameters; 

an annotator to store the differential information using the settable parameters in the 
video bit stream by extending the original bit stream format only if the differential 
information indicates a change from the current state of the auxiliary information. 

24. (Original) The apparatus of Claim 16, wherein the auxiliary information 
comprises additional non-visual information regarding camera geometry and 
identification for the video frames. 

25. (Original) The apparatus of Claim 16, wherein the auxiliary information 
comprises video processing information regarding descriptions and camera positions for 
the video frames. 
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